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ABSTRACT 

The  Decision-Making  Evaluation  Facility  for  Tactical  Teams,  or 
DEFTT,  is  a  portable  testbed  system  designed  to  provide  a 
simulation  environment  for  conducting  tactical  decision-making 
research.  Teams  engage  in  nine  vignettes  which  provide  the  context 
for  studying  ‘  naturalistic”  decision  making  in  a  realistic 
environment.  A  primary  concern  was  to  maintain  the  fidelity  of  the 
Navy  combat  information  center  (aboard  ships)  for  six-member 
tactical  teams  under  relatively  stressful  conditions.  Six  consoles  and 
an  experimenter  control  station  are  electronically  linked,  and  actions 
taken  by  each  team  member  are  recorded  as  they  respond  to  highly 
complex  decision  events. 
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A  brief  description  is  given  of  the  (1)  research  objectives  (including 
some  of  the  psychological  and  engineering  principles  guiding  the 
research);  (2)  general  methodological  approach;  (3)  hardware  and 
software  used;  and  (4)  some  development  issues  related  to 
operationalization  of  such  a  research  testbed. 


INTRODUCTION 


This  paper  covers  a  brief  overview  of  the  Tactical  Decision  Making  Under  Stress  (TADMUS) 
program,  sponsored  by  the  Office  of  Naval  Technology,  being  conducted  as  a  joint  laboratory 
effort  at  Naval  Command,  Control  and  Ocean  Surveillance  Center  (NCCOSC),  San  Diego,  and 
Navy  Training  Systems  Center  (NTSC),  Orlando;  then  describe  the  Decision-Making  Evaluation 
Facility  for  Tactical  Teams,  or  DEFTT,  and  some  of  the  related  development  efforts  and  issues 
related  to  conducting  this  research  program. 

BACKGROUND 


Complex  information  gathering  and  processing  systems  have  been  designed  to  aid  the  decision 
maker.  However,  these  systems  often  increase  the  decision  maker’s  burden  due  to  the  inherent 
system  complexity  and  the  failure  to  use  basic  principles  of  human  information  processing  in 
system  design.  Moreover,  these  systems  require  operators  to  perform  difficult  cognitive  tasks 
under  heavy  workloads.  In  engagements  involving  crisis  and  limited  objective  warfare  situations 
(CALOW),  which  may  represent  the  majority  of  future  anticipated  naval  conflicts,  the  decisions  to 
be  made  are  even  more  complcA  than  they  would  be  iti  full-scale  warfare.  Civilian  and  neutral 
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nation  resources  are  often  in  the  conflict  area.  Incoming  information  thus  carries  an  added  clemciu 
of  uncertainty.  In  these  situations,  interpretation  of  the  niles  of  engagement,  contact  identification, 
inferring  intent  from  identification  and  behavior,  and  the  shoot/no-shoot  decision  often  pose 
extremely  difficult  decision  problems.  In  extreme  cases  there  is  no  clear  cut  right  or  wrong 
answer  about  a  decision.  Rapidly  unfolding  events  result  in  severe  time  pressure  and  severe 
(often  catastrophic)  consequences  for  errors.  Current  battle  management  systems  are  not  optimized 
for  use  in  CALOW  situations  where  human  intervention  in  decision  making  is  required. 

The  objective  of  the  TADMUS  program  is  to  apply  recent  developments  in  decision  theory  and 
information  display  technology  to  the  problem  of  enhancing  tactical  decision  quality  under  stressful 
conditions.  The  products  of  this  research  will  be  demonstrated  in  the  context  of  anuair  warfare 
scenarios,  however,  general  principles  will  be  developed  that  will  be  applicable  to  other  warfare 
areas.  A  long-term  objective  is  to  produce  an  experimental  decision-suppon  system  with 
sufficient  robii'tness  to  permit  examiaution  of  tactical  decision  making  under  different  stresslul 
conditions.  Prototype  system  components  and  tools  will  be  evaluated  in  simulated  tactical 
environments,  initially  in  a  laboratory  setting,  and  later,  during  at-sea  exercises.  Development  of 
the  prototype  software  support  tools  will  be  based  on  decision -process  models  developed  by  Klein 
(1991),  situation  assessment  tools  developed  by  Noble  (in  press),  and  explanation-based  reasoning 
tools  developed  by  Hair  and  Pickslay  ( 1992).  Klein’s  models  are  based  on  decision  making  in 
“naturalistic”  settings  which  are  characterized  by;  high  time  pressure;  ambiguous,  incomplete  and 
erroneous  data;  action-feedback  loops;  ill-defined  goals;  and  frequent  shifts  in  goals.  According  to 
this  recent  approach,  expert  decision  makers  may  rely  on  well-developed  memory  representations 
to  guide  decision  making  in  new  (but  similar)  situations.  From  this  point  of  view,  the  decision¬ 
making  process  is  considered  to  be  intuitive  rather  than  analytical. 

A  second  product  of  this  task  will  be  a  set  of  general  principles  and  prototype  displays  for 
advanced  decision  support  systems  to  mitigate  human  information  processing  biases  and  to 
enhance  tactical  decision  quality  under  stress  (Barnett,  Perrin,  &  Walrath,  1992).  A  substantial 
body  of  research  has  demonstrated  that  humans  apply  a  limited  set  of  heuristics  to  simplify 
decision  making  in  complex  and  ambiguous  situations,  and  that  these  heuristics  can  result  in 
errors,  or  judgment  bias,  when  compared  to  normative  theory.  Research  will  be  conducted  to 
assess  whether  naval  personnel,  trained  and  experienced  in  antiair  warfare  (AAW)  operations, 
exhibit  these  biases  when  performing  their  duties  during  AAW  scenarios.  For  example,  the 
confirmation  bias  may  have  been  operating  when,  as  was  indicated  in  the  official  investigation  of 
the  Vincennes  incident,  console  operators  “distorted  data  flow  in  an  unconscious  attempt  to  make 
available  evidence  fit  a  preconceived  scenario”  (Parks,  1989).  Research  will  also  be  conducted  to 
examine  human-machine  interface  concepts  which  maximize  both  the  effectiveness  of  tactical 
decision  aids  under  stressful  conditions  and  the  efficiency  with  which  the  use  of  decision  support 
systems  can  be  trained.  Display  concepts  will  be  evaluated  for  their  ability  to  aid  the  decision- 
maker(s)  in  acquiring  and  maintaining  the  ability  to  extract  and  integrate  information  rapidly  and 
accurately  from  decision  support  systems  under  high  stress  loads.  Products  of  this  task  will 
include  display  principles  for:  predictive  displays,  situation  as.ses.sment,  option  generation, 
resolution  of  conflicting  or  ambiguous  information,  and  cognitive  consistency  among  team 
members. 

DECISION-MAKING  EVALUATION  FACILITY  for  TACTICAL  TEAMS  (DEFTT) 
LABORATORY 

The  ambitious  timelines  proposed  for  TADMUS  technical  development  mandated  use  of  an  off-the- 
shelf  tactical  simulator  for  the  experimental  phases  of  the  research.  A  survey  was  conducted  of  the 
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existing  simulation  systems  for  their  potential  applicability  to  'I'ADMUS.  lissential  technical 
capabilities  included  the  following  features:  (1 )  ctipability  to  run  complex  antiair  warfare  scenarios; 
(2)  multiple,  networked  stations  to  enable  configuration  of  a  Navy  combat  infomiation  center 
(CIC)  suite;  (3)  capability  to  measure  operator  performance  at  all  consoles,  including  console 
keystrokes  and  timestamping  of  operator  actions  for  later  synchronization  to  simulation  events;  and 
(4)  ease  of  scenario  authoring  and  simulation  control.  Additional  logistical  capabilities  included: 

(1)  modest  cost;  (2)  demonstrated  system  performance  and  reliability;  (3)  transportability;  (4)  an 
unclassified  database;  and  (5)  minimal  support  requirements. 

The  Navy’s  tactical  antisubmarine  warfare  (ASW)  interim  trainer  (TASWIT)  was  selected  with 
modifications  required.  TASWIT  is  a  personal  computer-(PC)  based  system,  developed  in  1984, 
for  tactical  ASW  team  training  which  has  been  well  received  by  the  fleet  and  has  considerable 
current  support  with  planned  upgrades.  Software  is  written  in  UNIX/C  and  is  transportable. 
Additional  features  include  the  ability  to;  (1)  present  complete  “ground  truth”  or  only  what  the 
given  operator  would  normally  see;  (2)  simulate  input  from  a  variety  of  sensors  and  selectively 
downgrade  them;  (3)  support  monochrome  or  color  displays  and  accommodate  large-screen 
projections;  and  (4)  rapid  manipulation  of  task  features  from  an  instructor  console.  The  required 
modifications  included  converting  the  ASW  database  to  an  AAW  database  and  adding  performance 
measurement  capabilities.  This  modified  TASWIT  system  is  called  the  Decision-Making 
Evaluation  Facility  for  Tactical  Teams,  or  DEFTT,  Laboratory. 

In  support  of  the  TADMUS  experiments,  DEFTT  provides;  (a)  a  multithreat,  multiplatform 
environment,  (b)  a  broad  range  of  sensor  inputs  and  tactical  information  that  replicates  the  type  of 
information  available  to  tactical  decision  makers,  (c)  a  scenario  generation  capability,  and  (d)  a  data 
collection  and  playback  capability.  To  gather  the  controlled,  experimental  data  needed  to  develop 
decision  performance  models  and  solutions  to  identified  problems,  the  DEFTT  laboratory  was 
developed  to  mimic  critical  features  of  an  Aegis  combat  information  center.  The  emphasis  on 
Aegis  combat  teams  and  systems  was  predicated  on  the  increasing  presence  of  these  systems  in 
Fleet  operations. 

The  DEFTT  system  is  comprised  of  six  computer  workstations  and  an  experimenter  control  station 
where  the  scenario  is  interactively  controlled  in  response  to  behaviors  by  the  team  members.  The 
six  workstations  simulate  the  following  positions  which  are  found  aboard  a  Navy  Aegis  cruiser; 
commanding  officer  and  tactical  action  officer,  both  of  whom  use  the  Aegis  display  system;  the 
antiair  warfare  coordinator,  tactical  information  coordinator,  and  the  identification  supervisor,  all  of 
whom  use  the  command  and  display  (C&D)  system;  and  the  electronic  warfare  supervisor  who 
uses  the  AN/SLQ-32  system  ( a  passive  receiver/analysis  system  for  radar  signals). 

DEFTT  SYSTEM  DESCRIPTION 

DEFTT  integrates  hardware  stations  and  software  modules.  The  major  hardware  stations  are  a 
Hewlett-Packard  (HP)  345  workstation  (referred  to  as  the  experiment  control  station  [ECS])  with 
an  experiment  control  software  module  and  six  Gateway  20(k)  386  personal  computers  with  Aegis 
display  system  (ADSL  command  and  decision  (C&D),  and  electronic  warfare  (EW)  software 
modules.  The  DEFTT  components  are  interconnected  through  a  local  area  network  (LAN)  as 
illustrated  in  figure  1 .  (LAN  is  hardware  and  software  used  to  interconnect  a  wide  variety  of 
normally  incompatible  processors,  operating  systems,  and  network  interfaces.) 
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Note:  Each  station  can  utilize  any  of  the  following  display 
formats:  C&D,  ADS,  or  EW. 


Figure  1.  DEFTT  System. 


Hardware  Architecture.  DEFTT  consists  of  the  following  three  major  components;  HP 
9000-series  Model  345CH  (HP  345)  computer  (1);  IBM- AT  type  386  personal  computers  (6);  and 
a  high  resolution  19-inch  monitor  ( 1 ).  Figure  2  illustrates  the  DEFTT  laboratory  configuration. 
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Note:  The  experimenter  control  station  (HP  345)  and  the  19-inch  monitor  are  co-located  in  a 
separate  game  control  room.  This  depiction  shows  five  of  the  six  workstations  and  the  large 
screen  display. 

Figure  2.  DEFTT  Laboratory. 

The  PCs  are  networked  together  with  the  ECS,  which  is  used  to  control  the  simulation  programs  in 
the  PCs.  Platform  position  and  status  data  are  calculated  in  the  ECS  and  sent  to  the  PCs  every  six 
seconds.  All  experiment  setup  functions  are  performed  at  the  ECS  with  specific  data  downloaded 
as  appropriate  to  the  PCs.  The  PCs  serve  as  selected  AEGIS  operator  stations,  allowing  operators 
to  perform  AEGIS  display  system  (ADS),  command  and  decision  (C&D),  and  EW  functions. 

Software  Architecture,  DEFTT  consists  of  the  following  software  components:  Experiment 
Control  Module,  Aegis  Display  System  (ADS)  Module,  Command  and  Decision  (C&D)  Module, 
and  EW  Module.  The  Experiment  Control  Module,  installed  in  the  ECS,  provides  for  generation 
and  control  of  scripted  TADMUS  scenarios  and  control  of  ADS,  C&D,  and  EW  module  functions. 

Each  PC  can  be  configured  as  separate  platforms  or  grouped  together  as  different  displays  on  the 
same  platform.  Each  station  is  configured  for  a  particular  display  at  the  ECS  during  scenario 
setup.  The  platform  and  display  configurations,  along  with  appropriate  data  sets,  are  transmitted  to 
the  operator  stations  from  the  ECS  during  the  startup  routine. 

Command  and  control  support  includes  the  two  main  computer  systems  in  the  Aegis  combat 
information  center:  C&D  and  ADS.  The  C&D  tactical  display  provides  simulated  C&D 
geographic  situation  (GEOS IT)  summary  displays,  selected  C&D  track  data  control  functions,  and 
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platform  control  functions  (dynamics,  sensor  control,  and  weapons),  i’he  ADS  tactical  display 
provides  simulated  ADS  displays  and  associated  control  functions.  'I'lie  FAV  tnoduic  provides 
simulated  AN/SLQ-32(V)3  countenneasures  set  electronic  support  mctisures  (FSM )  detection  and 
display  formats. 

DEFTT  SYSTEM  CAPABILITIES 

Experiment  Control  Capabilities.  The  experiment  director  controls  the  overall  operation  of 
DEFTT  through  the  use  of  the  DEFTT  ECS  (HP  345  computer,  function  keypad,  trackball,  and 
high  resolution  monitor).  The  experiment  director  perfomis  all  experiment  setup  and  experiment 
control  functions.  When  configured  in  the  experiment  mode,  the  experiment  director  has  the 
capability  to  generate  scenarios,  control  the  exercise  by  responding  to  subjects’  actions,  and  play 
back  a  specified  exercise. 

Scenario  Generation.  TADMUS  scenarios  establish  the  time-ordered  platform  events  for  each 
experiment  run.  The  experiment  director  has  the  option  to  use  an  existing  scenario,  modify  an 
existing  scenario,  or  develop  a  new  scenario.  DEFTT  supports  150  dynamic  platforms  per 
scenario.  These  dynamic  platforms  include:  own  ship;  surface  vessels  (friendly,  neutral,  and/or 
hostile  types);  own  ship-assigned  aircraft;  and  independent  aircraft  (friendly,  neutral,  and/or  hostile 
types).  In  addition,  up  to  150  tracks  per  scenario  can  be  entered  by  the  station  operators  and/or 
experiment  director  through  the  C&D  track  manager  function. 

Own  ship  is  a  CG  47-Class  cruiser  and  is  configured  with  the  CG  47-Class  antiair  warfare  (AAW) 
sensor  suite.  The  experiment  director  can  specify  aircraft  platforms  assigned  to  own  ship  including 
light  airborne  multipurpose  system  (LAMPS)  type  and  combat  air  patrol  (CAP)  aircraft.  The 
experiment  director  specifies  other  dynamic  platforms  to  be  used  in  the  scenario  including 
associated  platform  emitters.  A  DEF  lT  scenario  contains  the  following  initial  settings  for  all 
scenario  parameters:  data  link  reference  point  (DLRP)  position,  platform  type  and/or  name, 
platfoim  course,  platform  speed,  platform  initial  X/Y  position,  platform  bearing/range  to  DLRP, 
platform  altitude,  platform  classification,  platform  track  number,  platform  identification  friend  or 
foe  (IFF)  codes  and  transponder  status,  platform  emitter  types,  and  platform  modes  and  status. 

Playback.  Prior  to  running  another  exercise,  the  experiment  director  can  save  the  scenario  track 
history  for  playback.  The  playback  display  is  available  at  the  ECS  and  at  all  C&D  stations. 
Playback  replays  an  exercise  by  displaying  geographic  track  history  data,  including  NTDS 
symbols,  for  all  platforms  activated  at  any  time  in  a  scenario.  The  playback  can  run  in  real  time  or 
at  an  accelerated  time  rate  (2X  or  6X  real  time). 

Command  and  Decision  Geographic  Situation  (GEOSIT)  Display  Capabilities. 

O 

The  C&D  is  a  360  geographic  situation  display  that  presents  appropriate  navy  tactical  data  system 
(NTDS)  symbology  to  the  experiment  director  and  the  station  operators.  The  experiment  director 
and  the  station  operators  have  the  capability  to:  alter  display  radius  (range  scale),  change  display 
center,  enable/disable  range  rings,  enable/disable  display  of  friendly  tracks,  enable/disable  display 
of  sonobuoy  positions,  enable/disable  display  of  track  history  dots,  and  enable/disable  display  of 
course/spe^  vectors.  In  addition,  the  experiment  director  can  display  reported  positions  of 
contacts  and  actual  positions  of  hostile  contacts. 

Station  operators  are  provided  with  a  ball  tab  capability.  The  ball  tab,  which  is  controlled  by  a 
trackball,  can  be  used  by  the  operators  to  hook  and  sequence  tracks.  When  a  track  is  hooked,  the 
following  data  is  displayed  in  the  tabular  readout  area  (referred  to  as  the  character  readout  (CRO)): 
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hooked  track  category  (own  ship,  air  i  riend,  surl'ace  unknown,  and  so  lorih);  track  luiinbcr 
(system  track  number  (STN)  and/(,)r  common  track  stores  locator  (Cl'SL));  track  Cartesian 
coordinates;  track  latitude  and  longitude  (LAT/LONG);  track  course/speed:  track  altitude  and  height 
source;  track  identification  amplification  (strike,  tanker,  DD/DDG,  and  so  forth);  IFF-' data  (modes 
1-4);  and  track  bearing  and  range  from  ow-n  ship. 

Command  and  Decision/Track  Management  Capabilities.  The  experiment  director  at  the  lECS  and 
C&D  operators  perform  track  management  using  the  following  C&D  functions;  new  track  entry, 
old  track  update,  drop  track,  and  contact  classification.  C&D  station  operators  can  challenge 
contacts  on  any  EFF  mode.  Contacts  can  be  tracked  automatically  by  the  system  (including 
automatic  generation  of  NTDS  symbology),  or  manually  by  the  station  operators  (including  manual 
generation  of  NTDS  symbology). 

AEGIS  Display  System  Display  Capabilities.  The  ADS  is  a  geographic  presentation  of  a 
selected  portion  of  the  world  map.  The  ADS  station  operator  has  the  capability  to:  alter  display 
range  scale,  change  display  center,  define  and  display  tags,  enable/disable  display  of  course/speed 
vectors,  define  and  display  circles,  enter  general  reference  points,  define  lines  and  patterns,  and 
filter  C&D  track  symbology.  ADS  track  data  is  represented  by  appropriate  NTDS  symbology. 

The  ADS  station  operator  is  provided  with  a  ball  tab  capability.  The  ball  tab  is  controlled  by  a 
trackball.  Using  the  ball  tab,  the  operator  can  hook  and  sequence  tracks,  select  items  from  the 
CRO,  and  select  ADS  functions.  Data  for  a  hooked  track  is  displayed  in  the  CRO. 

Electronic  Warfare  Simulation  Capabilities.  The  AN/SLQ-32(V)3  simulation  provides 
polar  and  tabular  displays  of  ESM  contact  data  and  controls  for  launching  chaff.  Emitter 
detections  are  a  function  of  AN/SLQ-32(V)3  performance  characteristics  as  applied  to  a  simplified 
line-of-sight  (LOS)  equation.  At  each  time  step  (six  seconds),  emitter  detection  is  simulated  by 
comparing  the  positions  of  own  ship  and  the  emitter.  If  the  emitter  is  in  the  LOS  of  own  ship  and 
within  the  AN/SLQ-32(V)3  frequency  range,  the  emitter  is  detected  and  appropriate  symbology/ 
emitter  data  appears  on  the  display.  The  EW  display  provides  the  EW  station  operator  with  the 
following  data;  emitter  alerts,  threats  summary,  frequency  bands  enabled,  and  inhibited  alen 
status. 

Controls  are  provided  to  allow  the  EW  station  operator  to  control  the  following  EW  station 
functions:  polar  display  orientation  (true  or  relative);  alen  inhibit  status  (friendly,  all,  or  none); 
ESM  inhibit  status  (ttendly,  hostile  nonmissile,  or  unknown);  ESM  inhibit  sectors;  super  rapid 
bloom  offboard  chaff  (SRBOC)  employment;  and  ball  tab  position. 

Operator  functions.  During  a  scenario  run,  functions  performed  by  the  DEFTT  operators 
include:  Ship  control  -  engine  and  rudder  orders;  aircraft  control  -  course,  speed,  and  altitude; 
sensor  control  --  on,  off,  and  pulse;  weapon  control  —  torpedo  and  missiles;  and  track 
management. 

Communications  system.  A  communications  system  to  replicate  the  communication 
capabilities  available  aboard  ship  was  considered  an  essential  feature  of  the  DEFTT  laboratory 
as  communication  of  tactical  information  is  an  ongoing  and  critical  element  of  tactical  decision 
making.  Specific  capabilities  of  this  system  include  the  ability  to;  ( 1 )  replicate  on  and  off  ship 
communications  simultaneously,  (2)  allow  operator  monitoring  of  two  channels  simultaneously, 

(3)  record  all  channels  separately,  (4)  in.seri  background  noise,  and  (5)  configure  system  to  support 
opposing  team  play.  Additional  features  include  (1)  a  robust,  yet  flexible  and  easily  expandable 
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system,  (2)  indicators  for  all  stations  when  initiating  communications  on  various  channels.  (  3) 
transportability,  and  (4)  the  ability  to  operate  individual  stations  from  desk  mounted  controls 

The  internal  net  provides  communications  betw'een  the  six  team  members  and  is  monitored  by  tlie 
operations  director  and  the  radio/telephone  talker,  who  act  as  confaleraies  simulating  various 
people  with  whom  the  team  would  need  to  communicate  (internal  to  the  ship  and  off  ship,  e  g., 
battle  group  commander,  Saudi  combat  air  patrol  controller,  foreign  pilots,  etc.).  The  command 
execution  net  connects  ownship  command  level  decision  makers,  the  commanding  officer  and 
tactical  action  officer,  with  higher  level  command  external  to  the  ship.  The  antiair  warfare 
command  and  reporting  net  is  used  by  the  non-command  personnel,  i.e.,  AAWC,  TIC,  IDS.  and 
EWS  to  communicate  sensor  and  contact  identification  information.  The  military  air  distress  net, 
or  MAD,  used  by  the  tactical  information  coordinator,  and  the  international  air  distress  net,  or  1  AD, 
used  by  the  identification  supervisor,  are  both  used  to  issue  warnings  and  communicate  with 
foreign  aircraft.  All  communications  during  the  experimental  nins  of  the  scenario  will  be  recorded 
for  later  analysis. 

SCENARIO  DESCRIPTION 

The  scenario  to  be  used  for  TADMUS  was  intentionally  developed  to  be  highly  ambiguous  .so  that 
the  decision  maker  may  never  be  certain  about  the  threat  represented  by  the  contact  or  the  intent  of  a 
threatening  contact.  The  worldwide  proliferation  of  sophisticated  weaponry  presages  an 
increasing  intensity  and  danger  in  this  type  of  warfare,  and  the  2010  combat  system  upgrade  must 
take  this  important  driver  into  consideration.  Basic  requirements  for  the  scenario  and  the  nine 
decision  vignettes  were:  (1)  operation  in  shallow/confined  waters,  (2)  neutral  and  hostile  countries 
in  close  proximity,  (3)  modern  blue/gray  systems  and  weapons  among  neutral,  friendly,  and 
hostile  nations,  and  (4)  heavy  neutraVfriendly  traffic  in  the  vicinity.  These  last  three  features 
contribute  to  creating  a  scenario  with  many  ambiguous  situations,  i.e,,  where  loyalties  may  shift 
rapidly;  third  world  countries  possess  first  world,  high  technology  weapons;  and  tactics  are  ill- 
defined  for  the  situation. 

The  scenario  is  set  in  the  Persian  Gulf  where  several  of  the  nations  initially  classified  as  friendly 
have  reason  for  suddenly  changing  their  loyalties.  Thus,  the  continued  applicability  of  the  rules  of 
engagement  is  uncertain  and  the  intent  of  an  approaching  threat  from  any  one  nation  is  uncertain. 
Also,  the  wide  distribution  of  blue/gray  equipment  causes  uncertainty  as  to  the  national  ongin  of  a 
contact  (Riffenburgh.  1991).  The  vignettes  were  designed  to  be  independent  for  experimental 
control  with  the  intent  that  a  decision  in  one  vignette  would  not  influence  decisions  in  later 
vignettes.  The  scenario  includes  the  overall  structure  into  w'hich  the  nine  vignettes,  or  brief 
incidents,  are  incorporated.  This  overall  structure  is  comprised  of  a  mission  statement,  geopolitical 
background,  and  rules  of  engagement.  Each  vignette  follows  the  pattern  of  being  set  in  a  poorly 
defined  situation  where  one  or  more  threats  of  uncertain  origin  and  uncertain  intent  approach  either 
own  ship  or  the  ship  being  protected  and  do  not  respond  to  warnings.  The  team  must  decide  on  a 
sequence  of  responses  as  the  situation  evolves. 

Development  Issues.  Nine  decision  vignettes  were  developed  within  one  scenario  setting. 

Each  vignette  poses  a  cumulative,  critical  decision  event  which  the  decision  makers  must  resolve. 

A  major  issue  in  the  development  of  the  vignettes  was  the  decision  not  to  incorporate  an  “obvious 
ending”  to  each  of  these  decision  events.  Because  fidelity  to  tactical  problems  was  paramount,  the 
scenarios  were  built  to  remain  ambiguous  to  the  decision  maker.  For  example,  a  hostile  airplane 
may  be  engaged  and  weapons  fired,  but  whether  critical  battle  damage  was  generated  remains 
unknown  to  the  decision  maker  for  the  duration  of  the  scenario.  This  was  u.sed  to  mimic  the  type 
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of  situations  that  occur  in  real  wartiine  contingencies,  such  as  iX'scrt  Stonii,  vviicrc  battle  datiiage 
reports  often  arrive  hours  to  days  afv ,  die  engagement.  I  tus  also  obviates  the  problem  of ‘'game 
playing”  (trying  to  outwit  the  co'"''jteii  and  learning  effects  w  hich  may  inas  subsetjuent  decision 
vignettes.  This  tactic  on  our  o''’  ,  forced  us  to  provide  relatively  fast  [laced  decision  events,  so  that 
pondering  the  tactics  of  the  ‘last  incident”  are  kept  to  a  minimum. 

Given  this  constremt,  measurement  issues  are  very  complex,  'rypical  measures  of  the  number  of 
“kills”  tu-e  not  Darticularly  useful.  Recall  that  the  ultimate  objective  of  this  program  is  to  develop 
the  most  r .  "'rtive  decision  suppon  tools  for  tactical  decision  making  processes  in  CALOW 
situations.  Number  of  kills  becomes  inappropriate  as  a  measure  of  system  effectiveness. 
Therefore,  measures  that  capture  the  nature  of  the  situation  assessment  and  the  timely  response  to 
that  assessment  are  our  focus.  For  exatnple,  one  of  the  mission  objectives  may  be  to  maintain  air 
control  over  a  particular  air  corridor  under  complex  rules  of  engagement.  I  his  requires  immediate 
identification  of  all  radar  contacts  in  that  sector.  The  speed  and  accuracy  with  which  this  is 
accomplished  by  the  identification  supervi.sor  becomes  a  viable  process  measure  for  that  position; 
communication  of  this  information  to  relevant  team  members  then  becomes  a  team  coordination 
measure;  and  finally,  analysis  of  w'hat  action  was  taken  given  the  radar  contact  infonnation  under 
the  prescribed  mles  of  engagement  becomes  a  measure  of  team  effectiveness.  Latencies  and  errors 
in  this  process  are  alternative  measurements  that  combine  to  paint  a  picture  of  team  effectiveness. 

The  decision  support  prototypes  discussed  earlier  will  be  incorporated  into  the  DEFIT  and  tested 
for  their  effectiveness  in  enhancing  situation  assessment  and  decision  making  of  the  tactical  team 
under  varying  stressful  conditions.  It  is  expected  that  because  the  decision  vignettes  are  modular 
and  fairly  robust,  future  research  will  be  conducted  on  various  other  issues  of  individual  and  team 
tactical  decision  making  in  technologically  complex  settings. 

Scenario  Task  Demands  on  Team  Members. 

Antiair  warfare  decision-making  hierarchy.  Antiair  warfare  decision  making  aboard  an 
Aegis  cruiser  is  structured  hierarchically  with  four  levels  of  responsibility:  command, 
coordination,  supervision,  and  operation,  Ownship  command  includes  the  commanding  officer 
and  the  tactical  action  officer  who  sit  in  the  front  of  the  combat  information  center,  each  with  a 
large  screen  display  mounted  in  front  of  them  in  addition  to  their  computer  system  display.  The 
next  level  of  decision  making  involves  coordination,  which  involves  the  antiair  warfare  coordinator 
and  the  tactical  information  coordinator.  At  the  supervisory  level,  we  are  including  the 
identification  supervisor  and  the  electronic  warfare  supervi.sor.  These  six  positions  were  identified 
as  critical  for  the  types  of  low  intensity  conflict  decisions  we  are  interested  in  inve.stigating. 

Crisis  and  Limited  Objective  Warfare.  The  types  of  decisions  which  will  be  the  focus  of 
our  efforts  include  determining  threat  identification,  threat  intent,  and  application  of  the  rules  of 
engagement  under  conditions  of  crisis  and  limited  objective  warfare  (CALOW).  lOespite  the  fact 
that  the  primary  design  criteria  for  navy  .ships  are  based  on  the  support  of  a  major  conflict  on  the 
high  seas,  CALOW  situations  are  expected  to  represent  a  major  proportion  of  future  naval 
operations.  CALOW  situations  are  characterized  by  their  own  unique  problems  and  system 
stresses,  particularly  within  the  information  processing  and  decision-making  domain.  Key 
characteristics  of  CALOW  are;  (1)  uncertain  motivations  of  various  forces  in  the  area;  (2) 
operating  under  complex  rules  of  engagement;  (3)  the  presence  of  friendly,  neutral,  and  suspect 
shipping  and  aircraft;  (4)  risk  of  damage  to  own,  friendly,  and  neutral  elements  must  be  held  low 
de.spite  ongoing  hostile  activities;  and  (5)  the  worldwide  proliferation  of  sophisticated  weapons 
adds  to  the  complexity  of  the  problem.  These  types  of  contlicts  add  many  layers  of  complexity  to 
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the  deeision making  problem  by  inehiding  several  levels  ot  threat,  multiple  combinations  ot 
engagement  rules,  and  an  extremely  high  level  of  uncertainly  about  the  intentions  of  contacts. 
When  decisions  Itave  to  be  made  within  extremely  short  time  windows,  with  varying  degrees  of 
degraded  and/or  ambiguous  infonnation,  and  with  ROEs  that  have  not  covered  all  the  potential 
situations,  the  process  of  decision-making  can  break  down. 

Task  Demands.  The  scenarios  require  a  considerable  amount  of  effort  to  (1)  deconflict  contacts 
of  interest  from  background  clutter  tracks,  then  (2)  detennine  the  identity  and  intent  of  the  contacts 
of  interest.  Additionally,  the  team  is  required  to  apply  the  complex  rules  of  engagement 
throughout  the  vignette.  While  these  vignettes  do  not  necessarily  call  for  an  engagement  decision, 
there  tire  several  instances  where  engagement  criteria  are  met  under  the  rules  of  engagement;  yet  in 
some  instances,  the  appropriate  response  would  be  to  not  engage.  Considerable  communications 
are  also  required  of  team  members.  These  include  communications  external  to  the  team  to  issue 
warnings  to  approaching  air  contacts.  For  example,  the  team  can  issue  a  warning  to  an 
approaching  aircraft  over  the  International  Air  Distress  Net.  Other  external  communications  are 
required  to  off  ship  assets,  such  as  the  officer  in  tactical  command  (the  Battle  Group  Commander), 
the  foreign  combat  air  patrol  aircraft  controller,  and  other  off  ship  information  sources.  Internal 
communications  are  required  among  team  members  to  clarify  ambiguous  data  and  to  share 
information.  In  general,  very  little  support  from  off  ship  as.sets  is  provided,  thus  forcing  multiple 
ownship  decisions. 
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